The Effect Size: Beyond Statistical Significance

The branch of statistics known as inferential statistics tries to find out information about a population by studying a representative sample of that population. The main tools of inferential statistics are the statistical significance tests. Every statistical significance test includes two hypotheses; the null hypothesis (H0) and the alternative hypothesis (H1). The null hypothesis assumes that there are no differences between the estimators under study. On the other hand, the alternative hypothesis supposes the existence of differences between them. These tests are conceived to demonstrate that the alternative hypothesis is the true one; these tests never demonstrate anything about the null hypothesis. The significance level (α) is the probability of being wrong when the alternative hypothesis is accepted. Since statistical significance tests try to demonstrate that the alternative hypothesis is the true one, the significance level is always defined at the beginning. For most experimental and observational studies it is equal to 0.05. For a statistical test, when the alternative hypothesis is accepted, the probability of taking the correct decision must be indicated. This probability is called P and must be lower than the significance level (α). Nevertheless, assuming that the conclusion is that a significant difference does exist, the P value does not inform about the magnitude of this difference. For example, the statistic t (t ‐Student test) and its associated P value, only allows to affirm that the difference between two means is significant, but it does not inform about the importance of the difference. There are other weakness to taking into account only the value of the statistic under study (t for the t‐Student, for example), because this value depends on the mean value and especially on the sample size (and also on the variance in the case of the t‐ Student test). Some years ago, in other scientific areas, especially in psychology the importance of the magnitude of the difference was described and designed as the effect size (1). Currently, the Publication Manual of the American Psychological Association, in its sixth edition, provides guidance on reporting effect sizes (2), emphasizing the need of reporting it systematically as a complement of the P value. The effect size is considered an essential complement of the statistical significance test when a significant difference is found, because it allows to know the relevance of the difference and discerning between the statistical significance of a test and its practical importance. This is especially important when the groups under study are formed by a great number of items, because in these cases it is more frequent to find statistical significant differences (See example 1). The Journal of the International Federation of Clinical Chemistry and Laboratory Medicine